Video coding has conventionally focused on improving video quality at a particular bit rate. With the rapid growth of network video applications, such as Internet streaming video, there is an impetus to optimize the video quality over a range of bit rates. Further, because of the wide variety of video servers and varying channel connections, there has been an interest in determining the bit rate at which the video quality should be optimized.
The variation in transmission bandwidth has led to the idea of providing fine granularity scalability (FGS) for streaming video. FGS coding is used, for example, in MPEG-4 streaming video applications.
The use of FGS encoding and decoding for streaming video is described in ISO/IEC JTC1/SC 29/WG 11 N2502, International Organisation for Standardisation, “Information Technology-Generic Coding of Audio-Visual Objects—Part 2: Visual, ISO/IEC FDIS 14496-2, Final Draft International Standard,” Atlantic City, October 1998, and ISO/IEC JTC1/SC 29/WG 11 N3518, International Organisation for Standardisation, “Information Technology-Generic Coding of Audio-Visual Objects—Part 2: Visual, Amendment 4: Streaming video profile, ISO/IEC 14496-2:1999/FPDAM 4, Final Proposed Draft Amendment (FPDAM 4),” Beijing, July 2000, the contents of which are incorporated by reference herein.
As described in an article by Li et al. entitled “Fine Granularity Scalability in MPEG-4 Streaming Video,” Proceedings of the 2000 IEEE International Symposium on Circuit and Systems (ISCAS), Vol. 1, Geneva, 2000, the contents of which are incorporated by reference herein, the encoder generates a base layer and an enhancement layer that may be truncated to any amount of bits within a video object plane (VOP). The remaining portion preferably improves the quality of the VOP. In other words, receiving more FGS enhancement bits typically results in better quality in the reconstructed video. Thus, by using FGS coding, no single bit rate typically needs to be given to the FGS encoder, but only a bit rate range. The FGS encoder preferably generates a base layer to meet the lower bound of the bit rate range and an enhancement layer to meet the upper bound of the bit rate range.
The FGS enhancement bitstream may be sliced and packetized at the transmission time to satisfy the varying user bit rates. This characteristic makes FGS suitable for applications where transmission bandwidth varies. To this end, bit plane coding of quantized DCT coefficients is used. Different from the traditional run-value coding, the bit plane coding is used to encode the quantized DCT coefficients one bit plane at a time.
In FGS, the enhancement layers are inherently tightly coupled to the base layer. Without appropriate time stamping on decoding and presentation, the decoding process will consume more memory than may otherwise be required. Additional memory leads to increased decoder costs, size and reduced efficiency of decoders, and may hinder the development of a standardized protocol for FGS. The problem is particularly pronounced with FGS temporal scalability (FGST), as the enhancement structures may include separate or combined enhancement layers for FGS and FGST. There is therefore a need to provide an apparatus and method for time stamping in a manner that helps to conserves memory requirements in an FGS system.